1. Field of the Invention
The present invention relates to a method and apparatus for outlining and aligning a face in face processing.
2. Description of the Related Art
Face alignment plays a fundamental role in many face processing tasks. The objective of face alignment is to localize the feature points on face images, such as the contour points of eyes, noses, mouths and outlines. The shape and texture of the feature points acquired by the alignment provide information for applications such as face recognition, modeling, and synthesis.
There have been many studies on face alignment in the recent decade, most of which were based on Active Shape Model (ASM) and Active Appearance Model (AAM) methods. In these methods, local or global texture features are employed to guide an iterative optimization of label points under the constraint of a statistical shape model.
Both ASM and AAM use a point distribution model to parameterize a face shape with a Principle Component Analysis (PCA) method. However, the feature model and optimization strategy for ASM and AAM are different. ASM introduces a two-stage iterative algorithm, which includes: 1) given the initial labels, searching for a new position for every label point in its local neighbors that best fits the corresponding local one-dimensional profile texture model; and 2) interpreting the shape parameters that best fit these new label positions. AAM is different from ASM in that it uses a global appearance model to directly conduct the optimization of shape parameters. Due to the different optimization criteria, ASM performs more accurately on shape localization, and is relatively more robust to illumination and bad initialization. However, the classical ASM method only uses a vector gradient perpendicular to the contour to represent the feature, and characterizes it with PCA. Since this one-dimensional profile texture feature and PCA are so simple, the classical ASM method may not be sufficient to distinguish feature points from their neighbors. Therefore, the classical ASM technique often suffers from local minima problem in the local searching stage.
Many different types of features, such as Gabor, Haar wavelet, and machine learning methods, such as Ada-Boosting and k-NN, have been employed to replace the gradient feature and simple gaussian model in the classical ASM methods and improve the robustness of the texture feature. Further, different methods of optimization, such as weighted least-square, statistical inference, and optical flows, have been carried out to improve the efficiency of convergence. However, these methods have not been sufficient to make face alignment with an accurate local texture model, which can be generalized to large data sets, practical for use.
Furthermore, the methods described above do not pay sufficient attention to the initialization of the alignment, which affects the performance of the alignment. Many face alignment processes begin with face detection, in which the face to be aligned is first detected. However, face detection algorithms only give a rough estimation of the position of a face region, and in many cases it may be difficult to align facial shapes starting from the rough estimation, due to the fact that face images may vary greatly due to differences in face shape, age, expression, pose, etc., which may cause errant initialization. Therefore, it may be difficult to estimate all the feature points properly in initialization. With a bad initialization, the iterative optimization of both ASM and AAM may get stuck in local minima, and the alignment may fail.
There have been efforts trying to solve face alignment using face detection techniques. Boosted classifiers, which have been widely used to recognize the face pattern, may be used to recognize smaller texture patterns for every facial feature point. See Zhang et al., “Robust Face Alignment Based on Local Texture Classifiers” in Proceedings of ICIP (2005). There has also been previous works employing hierarchical approaches. One of these methods includes a hierarchical data driven Markov chain Monte Carlo (HDDMCMC) method to deduce the inter-layer correlation. See Liu et al., “Hierarchical Shape Modeling for Automatic Face Localization” in Proceedings of ECCV (2002). Another method uses multi-frequency Gabor wavelets to characterize the texture feature. See Jiao et al., “Face Alignment Using Statistical Models and Wavelet Features” in: Proceedings of CVPR (2003). Still other methods use a multi-resolution strategy in their implementations, most of which simply take the alignment result in the low-resolution images as the initialization for high-resolution images.